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ABSTRACT 

As a result of the development of two computer 
statistics laboratories in the psychology department at New York’s 
Brooklyn College, a project was undertaken to develop and implement 
computer program modules in undergraduate and graduate statistics 
courses. Rather than use the technology to merely make course 
pr esentat ions more exciting, the project undertook to redesign course 
formats entirely. The computer program modules are designed to 
minimize hand calculations, allowing students to spend more time 
mastering concepts, and to allow instructors to utilize a mastery 
style evaluation of the students’ performance. In addition, the 
modules respond to students' requests for more examples by providing 
them with randomly generated problems and allow students to take 
exams as many times as they want. On-line student monitoring informs 
instructors of students' performance and provides students with 
immediate feedback on exams. Advantages of using the computer-based 
protocol include that it is highly flexible, gives instructors more 
time to provide students with individualized attention, and promotes 
active learning by allowing students to use the programs at their own 
pace and time. Disadvantages include the lack of sophistication in 
both generating test questions and grading and overcoming students' 
computer phobia. Includes sample module screen-shots. (TGI) 
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ABSTRACT 

It is the authors’ experience that students in introductory statistics courses tend to focus on rote calculations rather 
than the theories upon which statistical procedures were created. Using computer program modules to enhance 
lectures, the authors intend to create a class structure wherein calculations and rules could be easily obtained via 
computers. This permits more time for instructors to teach the underlying assumptions, characteristics and theories 
of each statistical test Furthermore, through computer generated quizzing and mastery-style evaluation of student 
performance, the instructors can monitor class progress, and modify their teaching techniques accordingly. 



INTRODUCTION 

At BrooKlyn College, the Psychology Department is 
currently in the process of setting up two Statistics 
laboratories for undergraduate and graduate 
students. The classrooms house 34 and 21 IBM PC 
compatibles respectively and were formed with the 
goal of teaching statistics more effectively and 
efficiently. In anticipation of the labs, we have been 
discussing, planning and developing a number of 
program modules that could be incorporated into the 
graduate and undergraduate courses. This paper 
briefly outlines the latest developments in the course 
designs and the program modules that we have 
started to use this semester. We will focus on the 
undergraduate statistics course because many of the 
following ideas were formed with the needs of 
undergraduate students in mind. 

HISTORY 

In 1993, the chairman of the Psychology department 
at BrooKlyn College approached Ki-Young Rah and 
asKed him to start experimenting with various ways 
to incorporate computers into the undergraduate 
statistics course. The psychology department had 
just received funds to set up two computer labs for 
the graduate and undergraduate statistics courses. 
Although many ideas had been proposed ranging 
from computerizing the lectures to virtual classrooms, 
they could not be tested nor improved due to the 




limited resources available at the time. 

Rah decided to taKe a conservative approach 
by taKing into account the limited student access to 
computers at the time. First, computer assignments 
using a spreadsheet program were utilized as extra- 
credit assignments. These assignments gave the 
students hands-on experience with various statistical 
theories. Second, because of the constant request 
for more examples, a program was written which 
randomly generated problems with data and 
answers. Finally, the students were given access to 
a freeware tutorial which was downloaded from the 
Internet. Overall, the students were very receptive to 
these additions. 

Two years later, with the impending 
completion of the statistics labs, the authors sat 
down to discuss how computers could be used to 
enhance undergraduate statistics courses given the 
increased student access to computers. 

PHILOSOPHY 

When we began to discuss how to taKe full 
advantage of the new facilities, we decided to shift 
our focus from using technology as a mere 
enhancement of lectures to restructuring the course 
itself. Enhancing lectures, especially for a dry 
subject matter such as statistics, can be of great 
value. Points can be illustrated with color graphs, 
animation, video and sound. Such enhancements 
may draw the student’s attention and increase the 
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likelihood that the point will be remembered. 
Unfortunately, such multimedia presentations still 
have the same inherent flaw as any form of lecturing. 
It is extraordinarily difficult to pace and present 
lectures that will satisfy all of the students. Any 
lecture format always results in some students 
becoming very bored, and others completely lost. 

In addition, using technology introduces two 
additional problems. First, the presentation may 
effectively entertain the students, but it may also 
serve to divert their attention. That is, they may miss 
the vital information that is given within the 
presentation. Second, by using video clips, slides 
and sound, flexibility is limited. One of the main 
reasons why blackboards are still used today is 
simply because they provide the greatest flexibility. 
It is difficult to edit a video clip or reshuffle the order 
of a computer slide show during a lecture. 
Furthermore, it is difficult to justify the amount of time 
taken to develop a multimedia presentation when 
conventional lecturing style could basically get the 
same points across, although perhaps in a less 
entertaining way. 

Taking note of such problems, we decided to 
shift our focus from presenting a more exciting, 
dynamic presentation to changing the format of the 
course itself. We began by defining our goals. What 
do we, as instructors, want the students to take away 
from the course? Our main goal, even when using 
conventional lecturing styles, has always been to 
teach students how to organize, analyze and interpret 
data accurately. To achieve this goal, the students 
need to: 1) know how and when to use various 
statistical techniques, 2) understand the theories 
behind the techniques and 3) develop a "number 
sense". 

Our experience has been that students tend 
to focus more on rote calculations and following 
steps rather than on the underlying theories of the 
statistical techniques. In other words, the students 
gained "declarative" rather than "procedural" 
knowledge. 

Many students come into the course 
dreading the next few months because of their fears 
of mathematics. They know that, in a typical 
professional setting, statistical analyses are mostly 
done with the help of computers. They argue that 
there is little need to know the formulas to correctly 
analyze data. 

Although this may be the case, many may 
argue that one loses some understanding of statistics 
when analyzing data strictly with computer 
assistance. We agree with this concern. Doing 
calculations by hand seems to better promote 
understanding of the mathematical bases for 



statistical tests. Students are better able to grasp the 
notion that a variance refers to “an average squared 
deviation from the mean" if they perform the 
calculations by hand. In addition, by performing 
enough calculations, they are able to estimate what 
the test statistic should look like by examining the 
data. This ability to predict what the final outcome 
should look like is what we refer to as "number 
sense." It is a very important ability because it allows 
the student to more accurately interpret results from 
a statistical package by performing error checking. 
As such, we have chosen to greatly de-emphasized 
the hand-calculation component of the course. 

With our goals in mind, we next focused on 
the immediate needs of the students. Every 
semester, a number of students make the same 
requests: more examples, more chances to prove 
comprehension of the material, and access to tutors. 
We endeavored to meet the first request by allowing 
the students to generate their own examples and 
calculations. For the second request, we decided to 
redesign the course to allow the students to take the 
final exam as many times as they wished. For the 
third request, the course was redesigned to be much 
more student oriented. 

INSTRUCTION 

At Brooklyn College, we are allotted three hours of 
lecture and four hours of laboratory time per week. 
As our course design stands now, we use the first 
month or two to lecture to the students. The 
laboratory time is used to teach the students how to 
use the computers and our software modules. After 
the lecturing has been completed, the lecture and 
laboratory time is combined to form one large class. 
This is the time period within which the students may 
practice, test their knowledge, or take exams at their 
own leisure. The instructors spend this time giving 
individualized assistance to the students when 
needed. This format was conceived to maximize the 
individualized help that we wished to achieve. 

The students are given access to a number 
of programs that provide examples, explanations and 
the ability to explore theoretical issues. EXPSTAT 
generates random studies, data and answers. The 
students may generate a random example and test 
themselves on the calculations or design topics. 
Similarly, the instructor can use EXPSTAT to clarify 
lectures through examples. A number of tutorials on 
various theoretical topics are also available for the 
students to go through on their own time. This 
allows students to review topics interactively at a 
pace with which they are comfortable. Lastly, a set 
of spreadsheet pages (QPWstat) were created that 



provide analyses and graphs. These spreadsheet 
pages are used to illustrate various theoretical issues. 
This is achieved during lectures where immediate 
changes in graphs and test results can be pointed 
out as different types of data are entered. The 
spreadsheet pages are also used for assignments 
which allows us to direct the exploration process and 
emphasize the important theoretical concepts. For 
example, the idea of a Type II error can be 
elucidated by having the students randomly select 
two samples from two different populations. The 
students are then instructed to test whether the two 
samples differ significantly. By considering whether 
the observed result conforms to the expected 
outcome, the students are provided with a concrete 
example of Type II error. 

When designing the above mentioned 
programs we had to consider their ease of use. We 
were also careful to create an interface that was 
consistent and simple to learn. We opted to use the 
Microsoft Windows™ due to its GUI interface and the 
simplicity of its 'point and click" operations. The 
program that brings the above mentioned programs 
into a single package is called SKRstat (see Fig. 1). 

mastery-style evaluation 

In our experience at Brooklyn College, students who 
enroll in an undergraduate introductory statistics 
course tend to vary enormously in their educational 
backgrounds and interests. Furthermore, be ca use 
we typically teach the evening section of the course, 
there is also a wide age range. 

We believe the mastery style is the most 
effective and accurate method of evaluating a 
student’s knowledge of statistics. By allowing the 
students to take the final exam multiple times, we 
minimize the effects of various extraneous factors. 
Unfortunately, applying a mastery format dramatically 
increases the amount of work for the instructor and 
teaching assistant. However, if we make use of 
computers to generate random exams (or randomly 
select exams from a database) and grade them, then 
no extra work would be added. In fact, the 
evaluation would be much simpler and would require 
less effort. 

We have developed such a program, called 
SKRquiz (see Figure 2). The program uses the same 
user interface as SKRstat so that student expertise in 
using the program does not impact their grade. 
SKRquiz can present two types of questions. First, 
theoretical topics and simple calculation questions 
are asked in a multiple choice format. The program 
randomly selects a set number of multiple choice 



questions from a question bank written in text format 
and presents them. 

Second, experimental design questions and 
calculations are tested using the previously 
mentioned example generation engine (EXPSTAT). 
Various questions about the study design are asked, 
such as tail, hypotheses, and statistical test to be 
used. Then, the calculation screen is displayed in 
the same way as shown in SKRstat However, 
instead of presenting the students with the completed 
calculation steps, the students are required to fill in 
randomly selected missing values. 

The exams are timed and the program 
provides immediate feedback on how the students 
are doing and allows them to review their answers. 
They are also given the opportunity to review any 
past exams and print them to study from as needed. 
This allows the students to learn from their own 
mistakes. Before taking the final exam, the students 
are encouraged to take practice quizzes to familiarize 
themselves with the format and difficulty of the 
questions. Only their best final exam score is used 
(and other assignment grades) when determining 
their course grade. We believe that this will motivate 
the students to learn beyond the minimum necessary 
to do well in the course. 

ON-LINE STUDENT MONITORING 

The use of computers for teaching and evaluating 
provides a very convenient way to monitor a 
student’s performance. The use of these various 
programs and exams are all logged for each student. 
This informs the instructors in which areas students 
have the most problems and whether the exams 
were too easy or difficult. This design also gives 
early warnings of students who are performing very 
poorly but are hesitant to approach the instructors for 
assistance or guidance. 

The students are also able to monitor their 
own performances. The computer provides 
immediate feedback on the exams and allows for 
later review. Also, through the use of SKRstat, the 
students are able to recognize their areas of 
weakness. Through this self-monitoring the students 
are constantly aware of their progress. 

ADVANTAGES/DISADVANTAGES 

Advantages 

There are a number of advantages to using a 
computer based protocol. First, the instructor 
spends less time formulating questions and 
calculating answers. We have found that one of the 
most frustrating limitations in lecturing is the amount 



of time spent doing calculations and drawing graphs. 
It would be much more efficient and effective to use 
a computer to do the calculations and draw the 
graphs both during lectures and during the lab 
section when the students are generating their own 
examples. This frees up the instructor’s time to focus 
on more theoretical topics and answer individualized 
questions during the lab section. Also, the chance of 
making an error is decreased by having the 
computer do the calculations. 

The same advantage is found for the 
evaluation process. A mastery-style evaluation 
protocol is possible only because the computer is 
able to effortlessly generate as many exams as 
needed. The time needed by the instructor and/or 
teaching assistant to generate and grade the same 
number of exams is unrealistic. 

Second, the student can use these programs 
to study at their own pace, on their own time. Thus 
the student takes an active role in the learning 
process. This individualized learning allows both the 
strong and weak students to learn at their own pace 
without affecting other students. Similarly, the exams 
are taken at the convenience of the students and as 
often as they wish. Therefore, their grades will not 
be so strongly affected by various personal problems 
that may arise throughout the semester, such as 
illnesses, religious commitments, etc. The students 
are also less anxious about taking the exams 
because they know that there will be other 
opportunities to show their true levels of knowledge. 
Also, the constant feedback as to how the students 
are doing, serves to motivate them to study. 

Finally, the current design is highly flexible. 
Feedback from students’ performances allows the 
instructor to optimize the type of exams given. 
Furthermore, the program modules can be 
implemented in a variety of different course designs 
and lecture formats. Depending on how the students 
use the various available programs, one can 
implement different levels of computerization. 

Disadvantages 

The main disadvantage to using computer generated 
exams is the lack of sophistication in both the 
generation and grading. The multiple choice 
questions can be as complex and theory oriented as 
possible because they are generated by the 
instructor. However, the study related questions are 
very structured and limited in many ways. Free-form 
calculation, making interpretations and answering 
questions in either essay or short answer format is 
not currently possible. Also, by allowing the students 
to take the exams multiple times, we run the risk of 
the students memorizing the correct answers to the 



multiple choice questions. This is a major problem 
that can only be minimized by increasing the size of 
the multiple choice question database. The same 
lack of sophistication is present in tutorials. A tutorial 
can never be written that will cover all the questions 
that students will ask. That is why human instruction 
is still necessary. 

A second disadvantage is computer phobia. 
Students who have never touched a computer before 
may perform poorly on exams and assignments 
partly because of their discomfort with using 
computers. We have tried to minimize this by urging 
the students to use the various programs and take 
practice quizzes as soon as possible, but there may 
still be students who remain uncomfortable using a 
computer. 

Lastly, some may argue that using computers 
as teaching assistants takes away the interactivity 
between the student and instructor and between the 
students themselves. This loss of interactivity would 
probably be a serious disadvantage if computers 
take over the role of the lecturer. However, CASIP 
was specifically designed to maximize this human 
interactivity between instructor and student rather 
than reduce it. 

CURRENT OBSERVATIONS 

The Spring 1 996 semester marks the first semester 
within which we have implemented our course 
design. Although the course has only been 
underway for about one month, we have noted some 
successes with our format. Within one month’s time 
we have noted that the students have an increased 
comfort with the computers and the program 
modules. Furthermore, the students have expressed 
an increased interest in the subject material due to 
the highly interactive nature of the assignments and 
tutorials. We have also noticed that the students 
were spontaneously forming small groups in the form 
of collaborative learning. It would seem that having 
a common enemy (the computer) does tend to bring 
people together. We also noticed that there were 
many students who preferred to work alone. We 
believe that this option to work alone or together is 
an improvement over the formalized collaborative 
learning paradigm. 

There have been, as well, some pitfalls. For 
example, the students have been somewhat 
bewildered by the amount of information that is 
presented in the first month. They sometimes have 
trouble grasping the concepts, and other times 
complain that the lecture moves too fast. This is 
understandable because we have designed the 
course such that they would receive ail the 



theoretical and basic concepts in the first month or 
so. The reasoning behind this format was to 
maximize the individualized instruction time that the 
students would receive once the lecturing section of 
the semester is over. However, we have been 
persuaded by the students to extend the lecture time 
this semester, and we will be considering alternative 
organizations for the lectured material. 

The use of computers introduced a number 
of problems that we had not foreseen. Security for 
assignments and exams is a major consideration. 
Although, the student must sign in with a user id and 
password, it is not very difficult to copy or modify key 
files that determine their grades. In response to this 
new threat, stringent security measures were devised. 
Each of the students’ disks are collected at the end 
of every class and backed up on the instructor’s 
machine. The students are also restricted from doing 
the assignments and taking the exams outside of the 
classroom. Lastly, the students will be required to 
sign in when they decide to take the final exam. 

Another problem with computer use is 
accidentally deleting or overwriting files. A number 
of students have already managed to overwrite 
previous assignments that had been completed. The 
frustration is understandable and very difficult to 
rectify. Again, we hope that enough experience with 
computers will limit the number of such disasters. 

Because this is the first semester in which we 
have presented this format of teaching statistics, we 
did not expect it to be flaw-free. However, because 
our computer modules have been set up for maximal 
flexibility, we hope that we can correct any major 
flaws in the design for future semesters. 
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